A Hybrid Data Cleaning Framework Using Markov Logic Networks

With the increase of dirty data, data cleaning turns into a crux analysis. The accuracy limitation existing integrity constraints-based approaches results from insufficient rules. In this paper, we present novel hybrid framework on top Markov logic networks (MLNs), termed as ${\sf MLNClean}$ , which is capable learning instantiated rules to supplement constraints. consists two steps, i.e., pre-processing and two-stage cleaning . pre-processing step, first infers set probable according MLNs then builds two-layer MLN index structure generate multiple versions facilitate process. two-stage presents concept reliability score clean errors within each version separately, afterward eliminates conflict values among different using fusion Considerable experimental both real synthetic scenarios demonstrate effectiveness in practice.